WHAT IS CLAIMED IS: 



1 LA method for generating one or more audio elements, the method comprising 

2 the steps of: 

3 receiving user input; 

4 creating a first audio element in response to the user input; 

5 displaying a visual representation of the first audio element located spatially on a 

6 two-dimensional layout, wherein a display position of the visual 

7 representation is independent of audio data in the first audio element and 

tj 8 controllable by a user. 
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.. 5 1 2. The method of claim 1 , wherein the step of receiving user input includes 

• ! = w 2 receiving control signals, and wherein the step of creating a first audio element creates a 

; : f 3 data structure having a unique identification number field, and a spatial location field. 

. 

w 

.1 SB, 

; ?. 1 3 . The method of claim 1 , wherein the step of receiving user input includes 

2 receiving audio input signal, and wherein the step of creating the first audio element 

3 includes storing representations of the audio input signal in an audio data field of the first 

4 audio element. 

1 4. The method of claim 3, wherein the step of displaying includes displaying an 

2 audio indicator within the visual representation of the first audio element, the audio 

3 indicator corresponding to the audio input signal. 
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5. The method of claim 4, wherein the step of displaying increases the size of the 
visual representation of the first audio element and the audio indicator based upon a 
duration of the audio input signal. 



1 6. The method of claim 4, wherein the audio indicator is a waveform, a length of 

2 the waveform relating to a time measurement of the audio input signal and the height of 

3 the waveform relating to a measurement of energy or pitch in the audio input signal 

1 7. The method of claim 4, wherein the audio indicator is a color graph, colors in 

2 the graph relating to a measurement of energy or pitch in the audio input signal. 

1 8. The method of claim 3, wherein the step of displaying includes providing an 

2 audio gauge indicating the amplitude of the audio input signal. 

1 9. The method of claim 3, wherein step of displaying includes displaying a 



2 control button, the control button beginning the logging of the audio input signal if the 

3 audio input signal is not being logged, and the control button ending the logging of the 

4 audio input signal if the signal is being logged. 

1 10. The method of claim 3, wherein the step of receiving user input includes 

2 receiving a signal indicating an ordinal value, and wherein the step of creating a first 

3 audio element includes storing the ordinal value in an ordinal field of the first audio 

4 element. 
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1 11. The method of claim 1 , further comprising the steps of: 

2 receiving additional user input; 

3 creating a second audio element in response to the additional user input; 

4 displaying a visual representation of the second audio element located spatially on 

5 a two-dimensional layout, wherein the display position of the visual 

6 representation of the second audio element is independent of a temporal 

7 sequence or relationship to the first audio element. 

1 12. The method of claim 1 1 , further comprising the steps of: 

2 creating a link between the first audio element and the second audio element; and 

3 displaying a visual representation of the link as a connector between the visual 

4 representation of the first audio element and the visual representation of 

5 the second audio element. 

1 13. The method of claim 12, wherein the step of creating a link includes the steps 

2 of: 

3 creating a link object; 

4 storing a link to the first audio element and an interval within the first audio 

5 element in the linking object; and 

6 storing a link to the second audio element and an interval within the second audio 

7 element in the linking object. 
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1 14. The method of claim 1 , wherein the step of receiving user input comprises 

2 receiving an identification of a media object while receiving an audio input signal; and 

3 wherein the method further comprises the step of creating a link between the identified 

4 media object and the first audio element. 

1 15. The method of claim 14, wherein the identification of a media object includes 

2 a range in the media object, and the method further comprise the step of determining a 

3 range in the first audio element corresponding to when the identification of the media 
1 1 4 object was received, and wherein the step of creating a link stores a reference to the 

'•i tJ 

l i 5 identified media object, a reference to the first audio element, the range in the media 

I. Ll 

:i 5 6 object and the range in the first audio element. 



y 1 16. The method of claim 1 4, wherein the step of displaying includes displaying a 

l i 5 2 representation of the link. 

1 17. The method of claim 1, wherein the step of creating the first audio element 

2 includes the step of assigning the audio element an ordinal value. 

1 18. The method of claim 17, wherein the step of assigning the audio element the 

2 ordinal value sets the ordinal value to be one plus the highest ordinal value for an existing 

3 element. 
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19. The method of claim 17, wherein the step of displaying reflects ordinal values 
by showing connecting lines between successive audio elements. 



1 20. The method of claim 1, further comprising the steps of: 

2 creating a link between the first audio element and a media object; and 

3 displaying a visual representation of the link as a connector between the visual 

4 representation of the first audio element and a visual representation of the 

5 media object. 

21. The method of claim 20, wherein the step of creating a link includes the steps 
creating a link object; 

storing a reference to the first audio element and an interval within the first audio 

element in the linking object; and 
storing a reference to the media object and an interval within the media object in 
the linking object. 



1 22. A method for deleting audio elements comprising the steps of: 

2 receiving user input; * 

3 responsive to user input, identifying an audio element to be deleted; 

4 determining a visual representation corresponding to the identified audio element; 

5 removing the determined visual representation from display; 

6 deleting the identified audio element from memory; 
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7 determining audio elements that have an ordinal value greater than the identified 

8 audio element; and 

9 retrieving and decrementing the ordinal value of each audio element determined 
1 o to have an ordinal value greater than the identified audio element. 

1 23 . A method for reordering two or more audio elements, the method comprising 

2 the steps of: 

3 receiving user input; 

4 identifying a chosen set of audio elements to be reordered from the user input; 
!; I 5 identifying a new order for the chosen set of audio elements based on the user 

I Z 6 input; 

L J 

I § 7 determining a lowest ordinal value for the chosen set of audio elements; 

K! 

;; an 

i y 8 modifying an ordinal value of each of audio element in the chosen set of audio 

? 3 9 elements beginning with the lowest ordinal value and using the new order; 

't 10 and 

!; n 11 updating display of chosen set of audio elements to reflect the modified ordinal 

12 values. 

1 24. The method of claim 23, further comprising the step of storing the modified 

2 ordinal values for the chosen set of audio elements. 

1 25 . The method of claim 23 , wherein the step of determining a lowest ordinal 

2 value for the chosen set of audio elements further comprising the steps of: 

3 retrieving each audio element in the chosen set; 
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4 retrieving an ordinal value for each audio element in the chosen set from its audio 

5 element; and 

6 comparing the retrieved ordinal values to determine a lowest value. 

1 26. The method of claim 23, wherein the user input includes selecting a 



2 reordering mode and moving a cursor over visual representations corresponding to audio 

3 elements using a pointing device, and wherein the chosen set is determined by the visual 

4 representations over which the cursor crosses, and the new order is an order in which the 

5 visual representations are crossed. 

3 1 27. The method of claim 26, wherein the method further comprises a step of 

Z 2 displaying a feedback as to which visual representations the cursor has been moved 

KB 

1 3 across and the order in which the cursor was moved across them. 

Ml 

i 1 28. The method of claim 23, further comprising the steps of: 

3 2 determining whether there are any audio elements not chosen for reordering; 

3 if there are audio elements not chosen for reordering, 

4 identifying any unselected audio elements; 

5 determining an ordinal value of each unselected audio element; 

6 modifying ordinal values of the unselected audio elements that have an 

7 ordinal value greater than the lowest ordinal value to a modified 

8 order; and 

9 updating display of the unselected audio elements to reflect the modified 
10 ordinal values. 
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1 29. A method of associating audio elements with a media object, the method 

2 comprising the steps of: 

3 receiving user input identifying a media object and an audio element to be 

4 associated; 

5 creating a link object; 

6 storing a reference to the media object and the audio element in the link object; 

7 and 

8 displaying a representation of the link object with the representations of the media 

9 object and the audio element. 

1 30. The method of claim 29 further comprising the steps of: 

2 receiving user input identifying a range in the audio element; and 

3 storing the range in the link object. 

1 31. The method of claim 30 wherein the step of displaying further comprises: 

2 displaying a visual representation of the audio element; 

3 displaying a visual representation of the media object; and 

4 displaying a connector between the visual representation of the audio element at 

5 the range and the visual representation of the media object. 

1 32. The method of claim 29 further comprising the steps of: 

2 receiving user input identifying a range in the media object; and 

3 storing the range in the link object. 
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1 33. The method of claim 32 wherein the step of displaying further comprises: 

2 displaying a visual representation of the audio element; 

3 displaying a visual representation of the media object; and 

4 displaying a connector between the visual representation of the audio element and 

5 the range in the visual representation of the media obj ect. 

1 34. A method of outputting audio signals, the method comprising the steps of: 

2 displaying a visual representation of at least one audio element located spatially 

3 on a two-dimensional layout, the visual representation of the audio 

4 element including an audio indicator; 

5 receiving user input; 

6 identifying a first audio element based on the user input; 

7 retrieving audio data for the identified audio element; 

8 outputting the retrieved audio data; and 

9 highlighting sections of the audio indicator corresponding to a temporal ranges for 
10 which audio output has been provided. 

1 35. The method of claim 34, wherein the audio indicator is a waveform, a length 

2 of the waveform relating to a time measurement of audio data and the height of the 

3 waveform relating to a measurement of energy or pitch in the audio data. 

1 36. The method of claim 34, wherein the audio indicator is a color graph, colors 

2 in the graph relating to a measurement of energy or pitch in the audio data. 
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37. The method of claim 34, wherein the highlighting is shading. 



1 38. The method of claim 34, wherein the audio indicator is a waveform and 

2 highlighting is rendering the waveform in bold. 

1 39. The method of claim 34, wherein the audio indicator is a waveform and the 

2 highlighting is rendering the waveform in a different color from the visual representation 

3 of the first audio element. 

),* 

1 1 1 40. The method of claim 34, wherein the step of displaying includes displaying a 

i. y 2 plurality of audio elements, and wherein the method further comprises the steps of: 
™ 3 determining an ordinal number for the first audio element; 

ly 

;! R 4 determining whether there are any audio elements with an ordinal number greater 

!l Z 5 than the ordinal number for the first audio element; 

, 2 6 identifying a second audio element with an ordinal number greater that the ordinal 

Ml 
> w 

7 number for the first audio element; 

8 retrieving audio data for the second audio element; 

9 outputting the retrieved audio data for the second audio element; and 

10 highlighting sections of a waveform corresponding to a temporal range for which 

1 1 audio output has been provided. 
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1 41 . The method of claim 40, wherein the steps of retrieving audio data and 

2 outputting the retrieved audio data are performed after all the audio data for the first 

3 audio element has been output. 

1 42. The method of claim 34, further comprising the steps of: 

2 receiving a stop signal from the user; and 

3 terminating the outputting of the retrieved audio data. 

1 43. The method of claim 34, further comprising the steps of: 

2 receiving a create new audio element signal from the user; and 

3 terminating the outputting of the retrieved audio data. 

1 44. A method of joining audio elements comprising: 

2 receiving user input identifying a first and second audio elements to be joined; 

3 creating a new audio element; 

4 retrieving information from the first audio element and storing it in the new audio 

5 element; 

6 retrieving information from the second audio element and storing it in the new 

7 audio element; and 

8 deleting the first and second audio elements. 

1 45. The method of claim 44, wherein the information from the first audio element 

2 includes an ordinal value. 



-54- 

20412/06488/DOCS/1228853.8 



1 



2 



3 



46. The method of claim 45, further comprising the step of determining the 
ordinal value of the first audio element and using the determined ordinal value as the 
ordinal value for the new audio element. 



1 47. The method of claim 45, further comprising the step of reducing by one the 

2 ordinal value of any audio elements having ordinal values higher than that of the first 

3 audio element. 



, 4 1 48. The method of claim 45, further comprising the step of modifying the links 

} °\ 2 associates with the second audio element to be associated with the new audio element. 

*; SS' 
SS8 

: s 

i. y 

It ; 1 49. The method of claim 45, wherein the step of modifying the links includes 

2 modifying a reference to point to the new audio element and adjusting the range 

1*2 3 information. 

■ h 

'Iwf 
*; 

I'U 4 50. A method of joining audio elements comprising: 

5 receiving user input identifying a first and second audio elements to be joined; 

6 retrieving information from the second audio element; 

7 storing the retrieved information from the second audio element in the first audio 

8 element; and 

9 deleting the second audio element. 
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51. The method of claim 50, wherein the information from the second audio 
element includes an audio data signal. 



1 52. The method of claim 50, further comprising the step of reducing by one the 

2 ordinal value of any audio elements having ordinal values higher than that of the first 

3 audio element. 



1 53. The method of claim 50, further comprising the step of modifying the links 
! »' w 2 associates with the second audio element to be associated with the first audio element. 

Si S3 

la 

i: ;;;; 1 54. The method of claim 53, wherein the step of modifying the links includes 

K SB 

! ;> 

1 " 2 modifying a reference to point to the first audio element and adjusting the range 
j'i 3 information. 

f ^ 

f Q 1 55. A method of splitting an audio element, the method comprising the steps of: 

2 receiving user input identifying an original audio element to be split and a 

3 splitting point; 

4 creating a first audio element 

5 creating a second audio element; 

6 retrieving audio data for the original element; 
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7 storing audio data from the beginning to the splitting point in the first audio 

8 element, and the audio data from the splitting point to the end in the 

9 second audio element; and 
10 deleting the original audio element. 

1 56. The method of claim 55, further comprising the steps of: 

2 determining any links to audio data from the splitting point in the original audio 

3 element to the end; 

4 updating a link object to refer to the new audio object; and 

f! 3 5 updating the display of the visual representations of the original audio element, 

■: a? 

:: 6 the new audio element and the links to reflect the new association. 

i fj 
IRS 

i( y 1 57. A method of splitting an audio element, the method comprising the steps of: 

1 1 2 receiving user input identifying an original audio element to be split and a 

.: a; 

} ^ 

3 3 splitting point; 

'". as? 

f L) 4 creating a new audio element 

5 retrieving audio data for the original element; 

6 storing audio data from the splitting point in the original audio element to the end 

7 in the second audio element; and 

8 deleting the audio data from the splitting point to the end from the original audio 

9 element. 
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1 58. The method of claim 57, further comprising the step of displaying visual 

2 representations of the original audio element and the new audio element located spatially 

3 on a two-dimensional layout. 

1 59. The method of claim 57, further comprising the steps of: 

2 determining the ordinal value for the original audio element; 

3 storing a value of the determined ordinal value plus one as the ordinal value for 

4 the second audio element; and 

5 updating the display of the visual representations of the original and new audio 

6 elements to reflect the stored ordinal values. 

1 60. The method of claim 59, further comprising the steps of: 

2 determining any audio elements having an ordinal value greater that the ordinal 

3 value for the original audio element; and 

4 increasing by one the ordinal values of those determined elements having ordinal 

5 values higher than that of the original element. 

1 61. The method of claim 57, further comprising the steps of: 

2 determining any links to audio data from the splitting point in the original audio 

3 element to the end; 

4 updating a link object to refer to the new audio object; and 

5 updating the display of the visual representations of the original audio element, 

6 the new audio element and the links to reflect the new association. 
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1 62. A method for displaying media objects in conjunction with outputting audio 

2 data, the method comprising the steps of: 

3 identifying an audio element; 

4 retrieving audio data for the identified audio element; 

5 outputting the retrieved audio data; 

6 retrieving a link object referring to the identified audio element; 

7 retrieving a media object referred to by the link object; and 

8 displaying the media obj ect. 

1 63. The method of claim 62, wherein the link object identifies an audio range in 



i; S 2 the audio data of the audio element to which it refers, and wherein the step of displaying 

:s bp. 

i ;"h 3 is performed when an audio range specified in the link object is reached. 

il 



r -j 1 64. The method of claim 62, wherein the link object identifies a range in the 

C 3 2 media object to which it refers, and wherein the step of displaying, displays the range in 

3 the media object. 

1 65. The method of claim 62, wherein the link object identifies a range in the 

2 media object to which it refers, and wherein the step of displaying, displays the range in a 

3 separate window. 

1 
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